The influence of dopamine on prediction, action and learning

In this thesis I explore functions of the neuromodulator dopamine in the context of autonomous learning and behaviour. I first investigate dopaminergic influence within a simulated agent-based model, demonstrating how modulation of synaptic plasticity can enable reward-mediated learning that is both adaptive and self-limiting. I describe how this mechanism is driven by the dynamics of agentenvironment interaction and consequently suggest roles for both complex spontaneous neuronal activity and specific neuroanatomy in the expression of early, exploratory behaviour. I then show how the observed response of dopamine neurons in the mammalian basal ganglia may also be modelled by similar processes involving dopaminergic neuromodulation and cortical spike-pattern representation within an architecture of counteracting excitatory and inhibitory neural pathways, reflecting gross mammalian neuroanatomy. Significantly, I demonstrate how combined modulation of synaptic plasticity and neuronal excitability enables specific (timely) spike-patterns to be recognised and selectively responded to by efferent neural populations, therefore providing a novel spike-timing based implementation of the hypothetical ‘serial-compound’ representation suggested by temporal difference learning. I subsequently discuss more recent work, focused upon modelling those complex spike-patterns observed in cortex. Here, I describe neural features likely to contribute to the expression of such activity and subsequently present novel simulation software allowing for interactive exploration of these factors, in a more comprehensive neural model that implements both dynamical synapses and dopaminergic neuromodulation. I conclude by describing how the work presented ultimately suggests an integrated theory of autonomous learning, in which direct coupling of agent and environment supports a predictive coding mechanism, bootstrapped in early development by a more fundamental process of trial-and-error learning

An open reproducible framework for the study of the iterated prisoner's dilemma

The Axelrod library is an open source Python package that allows for reproducible game theoretic research into the Iterated Prisoner's Dilemma. This area of research began in the 1980s but suffers from a lack of documentation and test code. The goal of the library is to provide such a resource, with facilities for the design of new strategies and interactions between them, as well as conducting tournaments and ecological simulations for populations of strategies. With a growing collection of 139 strategies, the library is a also a platform for an original tournament that, in itself, is of interest to the game theoretic community. This paper describes the Iterated Prisoner's Dilemma, the Axelrod library and its development, and insights gained from some novel research.Comment: 11 pages, Journal of Open Research Software 4.1 (2016

Profound Chemopreventative Effects of a Hydrogen Sulfide-Releasing NSAID in the APC(Min/+) Mouse Model of Intestinal Tumorigenesis

Canadian Institutes of Health Research grant to JL

Closing the sensory-motor loop on dopamine signalled reinforcement learning

No description supplie

Granger causality analysis of fMRI BOLD signals is invariant to hemodynamic convolution but not downsampling

Granger causality is a method for identifying directed functional connectivity based on time series analysis of precedence and predictability. The method has been applied widely in neuroscience, however its application to functional MRI data has been particularly controversial, largely because of the suspicion that Granger causal inferences might be easily confounded by inter-regional differences in the hemodynamic response function. Here, we show both theoretically and in a range of simulations, that Granger causal inferences are in fact robust to a wide variety of changes in hemodynamic response properties, including notably their time-to-peak. However, when these changes are accompanied by severe downsampling, and/or exces- sive measurement noise, as is typical for current fMRI data, incorrect inferences can still be drawn. Our results have important implications for the ongoing debate about lag-based analyses of functional connectivity. Our methods, which include detailed spiking neuronal models coupled to biophysically realistic hemodynamic observation models, provide an important ‘analysis-agnostic’ platform for evaluating functional and effective connectivity methods